Web Survey Bibliography
Relevance & Research Question: More and more surveys use multiple modes, which supplement or replace traditional interviewer modes by web. In multi-mode questionnaire design, usually some consideration is given to mode-specific measurement error. Despite this consideration, however, these measurement effects are frequently unexpectedly large and hamper publication. For this reason, there is a strong incentive to better predict measurement effects. Measurement effects are determined by the interplay between characteristics of the questionnaire and characteristics of the respondents. In our research, we investigate the existence and utility of so-called questionnaire and respondent profiles, in which these characteristics are summarized, for predicting measurement effects. As a first research question, we ask whether questionnaires can be coded reliably on item characteristics that are suggested in the literature as influential in mode-specific measurement effects.
Methods & Data: We constructed a typology of item characteristics from the literature and applied it to a wide range of surveys; the Dutch Labour Force Survey of Statistics Netherlands and the core studies of the LISS panel of CentERdata. For all surveys, 16 item characteristics are coded by two main coders, while 7 of these 16 item characteristics that are assumed to be relatively influential in evoking measurement error are also coded by a third coder. Reliability diagnostics are derived for the various item characteristics.
Results: Analyses of the survey coding scores indicate a relatively low reliability for characteristics that literature suggested as influential in mode-specific measurement effects: Sensitivity to social desirable answers, potential presumption of filter question, emotional charge, centrality, and language complexity.
Added Value: It is investigated to what extent coding of questionnaires on its item characteristics is reliable and to what extent questionnaire profiles can be constructed based on this coding. Along with process data and register data that are linked to individual respondents who have filled out multiple questionnaires, the questionnaire and respondent profiles might shed light on the occurrence and scope of measurement effects for specific respondents and specific questionnaire characteristics over different survey modes.
Web survey bibliography - Toepoel, V. (27)
- Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not...; 2017; Toepoel, V.; Emerson, H.
- Data chunking for mobile web: effects on data quality; 2017; Lugtig, P. J.; Toepoel, V.
- Mobile-only web survey respondents; 2016; Lugtig, P. J.; Toepoel, V.; Amin, A.
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices...; 2016; Toepoel, V.; Lugtig, P. J.
- Doing Surveys Online ; 2016; Toepoel, V.
- The Effects of Adding a Mobile-Compatible Design to the American Life Panel; 2015; Toepoel, V.; Lugtig, P. J.; Amin, A.
- Higher Item Nonresponse Rates Caused by Slider Scales in Web Surveys; 2015; Toepoel, V.; Funke, F.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Investigating Response Quality in Mobile and Desktop Surveys: A Comparison of Radio Buttons, Visual...; 2014; Toepoel, V.; Funke, F.
- Informing panel members about study results; 2014; Scherpenzeel, A., Toepoel, V.
- Mixed-devices in a probability based panel survey. Effects on survey measurement error; 2014; Toepoel, V., Lugtig, P. J.
- Panel Conditioning in Difficult Attitudinal Questions; 2013; Binswanger, J., Schunk, D., Toepoel, V.
- Mobile devices a way to recruit hard-to-reach groups? Results from a pilot study comparing desk top...; 2013; Toepoel, V., Lugtig, P. J.
- Effects of Incentives in Surveys; 2012; Toepoel, V.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- How Visual Design Affects the Interpretability of Survey Questions; 2011; Toepoel, V., Dillman, D. A.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys; 2011; Das, M., Toepoel, V., van Soest, A.
- Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts.; 2011; Toepoel, V.
- Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?; 2011; Toepoel, V., Couper, M. P.
- Words, Numbers and Visual Heuristics in Web Surveys: Is There a Hierarchy of Importance?; 2009; Toepoel, V., Dillman, D. A.
- Pictures in Web Surveys; 2009; Toepoel, V., Couper, M. P.
- Panel Conditioning in Web Surveys: A Comparison between Trained and Fresh Respondents; 2009; Toepoel, V., Das, M., van Soest, A.
- Visual Heuristics and Answer Formats in Rating Scales; 2009; Toepoel, V. Dillman, D. A.
- Design effects in web surveys: comparing trained and fresh respondents; 2008; Toepoel, V., Das, M., van Soest, A.
- Can I use a panel? Panel conditioning and attrition bias in panel surveys; 2007; Das, M., Toepoel, V., van Soest, A.